{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 字符串处理综合案例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一、数据清洗综合案例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 案例背景\n",
    "我们有一份包含用户评论的数据，其中存在格式不一、含有无关字符、大小写混乱等问题。我们需要对这些数据进行清洗，以便后续分析。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# 创建一个包含脏数据的DataFrame\n",
    "data = {\n",
    "    'user_id': [1, 2, 3, 4, 5],\n",
    "    'comment': [\n",
    "        '  This is a GREAT product! I love it.  ',\n",
    "        'bad experience, will not buy again...  ',\n",
    "        '\tCould be better, maybe needs improvement.',\n",
    "        'AWESOME!!! #recommended',\n",
    "        np.nan\n",
    "    ],\n",
    "    'product_code': ['P001 ', ' P002', 'P003', 'P004', 'P005 '],\n",
    "    'rating': ['5 stars', '1 star', '3 stars', '5 stars', 'N/A']\n",
    "}\n",
    "df = pd.DataFrame(data)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 清洗步骤"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1 清理评论（`comment`）列"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. 转换为小写\n",
    "df['comment_cleaned'] = df['comment'].str.lower()\n",
    "\n",
    "# 2. 去除两端空白\n",
    "df['comment_cleaned'] = df['comment_cleaned'].str.strip()\n",
    "\n",
    "# 3. 替换特殊字符（例如#）\n",
    "df['comment_cleaned'] = df['comment_cleaned'].str.replace('#', '', regex=False)\n",
    "\n",
    "# 4. 链式操作\n",
    "df['comment_cleaned'] = df['comment'].str.lower().str.strip().str.replace('[^a-z0-9s]', '', regex=True)\n",
    "\n",
    "# 5. 处理缺失值\n",
    "df['comment_cleaned'] = df['comment_cleaned'].fillna('no comment')\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2 清理产品代码（`product_code`）列"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 去除空白\n",
    "df['product_code_cleaned'] = df['product_code'].str.strip()\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.3 提取评分（`rating`）列的数字"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 提取数字部分\n",
    "df['rating_cleaned'] = df['rating'].str.extract('(\\d+)').astype(float)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二、文本分析案例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 案例背景\n",
    "我们希望从用户评论中提取关键词，并分析评论的情感倾向。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 清洗后的数据\n",
    "df_cleaned = df[['user_id', 'comment_cleaned', 'rating_cleaned']].copy()\n",
    "df_cleaned"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 文本分析步骤"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1 提取关键词"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 查找包含'great'或'awesome'的评论\n",
    "positive_keywords = ['great', 'awesome']\n",
    "df_cleaned['is_positive'] = df_cleaned['comment_cleaned'].str.contains('|'.join(positive_keywords))\n",
    "df_cleaned"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2 情感分析（简单示例）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 根据评分判断情感\n",
    "def assign_sentiment(rating):\n",
    "    if rating >= 4:\n",
    "        return 'Positive'\n",
    "    elif rating <= 2:\n",
    "        return 'Negative'\n",
    "    else:\n",
    "        return 'Neutral'\n",
    "\n",
    "df_cleaned['sentiment'] = df_cleaned['rating_cleaned'].apply(assign_sentiment)\n",
    "df_cleaned"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 三、实际业务场景应用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 案例背景\n",
    "假设我们有一个销售数据集，其中`product_info`列包含了产品名称和版本号，格式为`产品名称-版本号`。我们需要将它们拆分为两列。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sales_data = {\n",
    "    'order_id': [101, 102, 103, 104],\n",
    "    'product_info': ['Laptop-v1.2', 'Mouse-v3.0', 'Keyboard-v2.5', 'Monitor-v1.0']\n",
    "}\n",
    "df_sales = pd.DataFrame(sales_data)\n",
    "df_sales"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 拆分产品信息"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用str.split()拆分列\n",
    "df_sales[['product_name', 'version']] = df_sales['product_info'].str.split('-', expand=True)\n",
    "\n",
    "# 移除版本号前缀'v'\n",
    "df_sales['version'] = df_sales['version'].str.removeprefix('v')\n",
    "df_sales"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结\n",
    "本教程通过三个综合案例，展示了Pandas字符串处理在数据清洗、文本分析和业务数据处理中的强大功能。熟练掌握这些技巧，可以极大地提高数据处理的效率。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
